1. Field of the Invention
The present invention relates to the field of computer graphics.
2. Related Art
Offline rendering algorithms have to a great extent conquered physically accurate photo-realism and complex synthetic shading. A result of over twenty years of research, these techniques all solve the lighting or rendering equation in some manner. See, the techniques in Blinn, J. F. and Newell, M. E., Comm. ACM 19:542-546 (1976); Cook, R. L., et al., xe2x80x9cThe Reyes image rendering architecture,xe2x80x9d in Computer Graphics (SIGGRAPH ""87 Proc.), vol. 21, Stone, M. C., ed., (July 1987), pp. 95-102; Debevec, P., xe2x80x9cRendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography,xe2x80x9d in SIGGRAPH 98 Conf. Proc., Cohen, M., ed., Annual Conference Series, ACM SIGGRAPH, Addison Wesley (July 1998), pp. 189-198; He, X. D., et al., xe2x80x9cA comprehensive physical model for light reflection,xe2x80x9d in Computer Graphics (SIGGRAPH ""91 Proc.), vol. 25, Sederberg, T. W., ed. (July 1991), pp. 175-186; Jensen, H. W. and Christensen, P. H., xe2x80x9cEfficient simulation of light transport in scenes with participating media using photon maps,xe2x80x9d in SIGGRAPH 98 Conf. Proc., Cohen, M., ed., Annual Conference Series, ACM SIGGRAPH, Addison Wesley (July 1998), pp. 311-320; Miller, G. S. and Hoffinan, C. R., xe2x80x9cIllumination and reflection maps: Simulated objects in simulated and real environments, in SIGGRAPH ""84 Advanced Computer Graphics Animation seminar notes (July 1994); Poulin, P. and Fournier, A., xe2x80x9cA model for anisotropic reflection, xe2x80x9d in Computer Graphics (SIGGRAPH ""90 Proc.), vol. 24, Baskett, F., ed., (August 1990), pp. 273-284; Veach, E. and Guibas, L. J., xe2x80x9cMetropolis light transport,xe2x80x9d in SIGGRAPH 97 Conf. Proc., Whitter, T., ed., Annual Conference Series, ACM SIGGRAPH, Addison Wesley (August 1997), pp. 65-76, and the rendering equation in Kajiya, J. T., xe2x80x9cThe rendering equation,xe2x80x9d Computer Graphics (SIGGRAPH ""86 Proc.), vol. 20, Evans, D. C. and Athay, R. J., eds., (August 1986), pp. 143-150.
A trade-off between speed and quality exists in off-line rendering and interactive rendering. The outstanding rendering challenge now becomes how to increase the performance of sophisticated shading algorithms without losing the advancements made in quality. This implies that many orders of magnitude in performance improvements must be found. Traditionally, this has been accomplished by vastly simplifying the approximations used in the shading and lighting equationsxe2x80x94resulting in a significant loss in complexity and quality.
Environment mapping is one method used to improve the realism of interactive rendering. As originally described by Newell and Blinn (Blinn, J. F. and Newell, M. E., Comm. ACM 19:542-546 (1976)), a simple environment map is used to quickly find reflections of distant objects from a perfectly mirrored surface. Other researchers refined this notion by generalizing the BRDF used, though some of these refinements lost the interactivity of simple environment mapping. See, Cabral, B., et al., xe2x80x9cBidirectional reflection functions from surface bump maps,xe2x80x9d in Computer Graphics (SIGGRAPH ""87 Proc.), vol. 21, Stone, M. C., ed., (July 1987), pp. 273-281; Greene, N., xe2x80x9cApplications of world projections,xe2x80x9d in Proc. Graphics Interface ""86, Green, M., ed. (May 1986), pp. 108-114; Miller, G. S. and Hoffman, C. R., xe2x80x9cIllumination and reflection maps:
Simulated objects in simulated and real environments, in SIGGRAPH ""84 Advanced Computer Graphics Animation seminar notes (July 1994); Poulin, P. and Fournier, A., xe2x80x9cA model for anisotropic reflection, xe2x80x9d in Computer Graphics (SIGGRAPH ""90 Proc.), vol. 24, Baskett, F., ed., (August 1990), pp. 273-284.
Another method used to bridge the gap between realism and interactivity is image based rendering (McMillan, L. and Bishop, G., xe2x80x9cPlenoptic modeling: An image-based rendering system,xe2x80x9d in SIGGRAPH 95 Conf. Proc., Cook, R., ed., Annual Conference Series, ACM SIGGRAPH, Addison Wesley (August 1995), pp. 39-46). Image based rendering (IBR) avoids solving the lighting equation during interactive rendering by warping existing photographs or images. These images can be thought of as radiance maps (Gershbein, R., et al., xe2x80x9cTextures and radiosity: Controlling emission and reflection with texture maps,xe2x80x9d in Proc. SIGGRAPH ""94 (Orlando, Fla., Jul. 24-29, 1994), Glassner, A., ed., Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, ACM Press (July 1994), pp. 51-58), and generalized to light fields (Levoy, M. and Hanrahan, P., xe2x80x9cLight field rendering,xe2x80x9d in SIGGRAPH 96 Conf. Proc., Rushmeier, H., ed., Annual Conference Series, ACM SIGGRAPH, ACM Press (August 1996), pp. 31-42) and lumigraphs (Gortler, S. J., et al., xe2x80x9cThe lumigraph,xe2x80x9d in SIGGRAPH 96 Conf. Proc., Rushmeier, H., ed., Annual Conference Series, ACM SIGGRAPH, ACM Press (August 1996), pp. 43-54). This works well for predefined scenes or images, but not for dynamically changing synthetic objects.
Recently, Debevec (Debevec, P., xe2x80x9cRendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography,xe2x80x9d in SIGGRAPH 98 Conf. Proc., Cohen, M., ed., Annual Conference Series, ACM SIGGRAPH, Addison Wesley (July 1998), pp. 189-198) combined captured environment maps and synthetic objects to produce compelling renderings with both synthetic objects and image based environments. His techniques do not work at interactive rates since he computes the lighting equation integration as he renders using RADIANCE (Ward, G. J., xe2x80x9cThe RADIANCE lighting simulation and rendering system,xe2x80x9d Proc. SIGGRAPH ""94 (Orlando, Fla., Jul. 24-29, 1994), Glassner, A., ed., Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, ACM Press (July 1994), pp. 459-572).
What is needed is an interactive photo-realistic rendering algorithm or system.
The present invention provides a method, system, and computer program product for reflection space image based rendering of an object at an interactive frame rate. A set of source radiance environment maps associated with a set of source viewing vectors are warped to create a destination radiance environment map associated with a destination viewing vector in a current frame. Blending and weighting operations can also be applied in creating the final destination radiance environment map. An object is then rendered with texture environment mapping from the destination radiance environment map.
According to one embodiment, at set-up, geometry information for a global set of source viewing vectors and a global set of source radiance environment maps are loaded. A viewpoint tracking loop is then carried out for a current frame. In the viewpoint tracking loop, a destination viewing vector that points along a view direction of the current frame is determined. A subset of source viewing vectors is then determined from the previously loaded global set of source viewing vectors. In one preferred example, the subset of source viewing vectors are a number of viewing vectors (e.g., three source viewing vectors) closest to the destination viewing vector. A set of corresponding weights for the subset of source viewing vectors is also determined.
A warping and blending loop is then performed iteratively on each source radiance environment map corresponding to the subset of source viewing vectors. In one embodiment, each source radiance environment map is warped and blended per texel according to a warping function to a create a destination radiance environment map.
In another preferred embodiment, a fast approximation is made which avoids warping per texel over the entire source radiance environment map. A tessellated mesh with a set of warped texture coordinates is generated. For example, in the case where the source radiance environment maps are sphere maps, the tessellated mesh is simply a tessellated disk. The set of warped texture coordinates at each vertex of the tessellated mesh represent a respective source radiance environment map texel which has been warped according to a warping function evaluated at the destination viewing vector. The mesh is then rendered (also referred to as drawn) using the respective source radiance environment map as a texture map to obtain a texture image. These texture images are also blended with weights corresponding to the respective source viewing vectors. The blended texture images are accumulated in a buffer (such as a frame buffer) to obtain a final destination radiance environment map. Control returns to the viewpoint tracking loop.
The final destination radiance environment map can be further normalized by weight if the weights were not previously normalized. The final destination radiance environment map is then loaded as environment map. An object or scene is then rendered with texture environment mapped from the final destination radiance environment map.
The present invention can be implemented in software, firmware, hardware, or in any combination thereof. The present invention further can be implemented in any graphics architecture or system that supports texture mapping and environment mapping. In one example, not intended to limit the present invention, a routine implementing the present invention is executed on a host processor using a graphics API or library, such as, OpenGL(copyright).
Thus, the present invention goes beyond the work of Debevec by pre-integrating the lighting equation to allow interactive renderingxe2x80x94in part by constraining the geometry of the global radiance environment maps. This introduces a view-dependence in the generated maps, which is overcome with a new form of IBR in reflection space. The result is interactive rendering of synthetic objects using a wide range of surface reflectances (including but not limited to those described by BRDFs) in arbitrary lighting environments.
There are at least two contributions made by the present invention. The first is the application of IBR techniques in reflection space. This is called reflection space IBRxe2x80x94even though the invention does not necessarily operate on normal images, nor is the result a rendered image. Second is our hybrid rendering algorithm. The algorithm of the present invention and the nature of the source radiance environment maps allows new surface reflectances (e.g., BRDFs) and new geometry not found in the original source images to be usedxe2x80x94thus extending the class of renderings possible with interactive IBR algorithms.
In sum, the present invention is an intermediate step toward interactive photo-realistic renderingxe2x80x94an algorithm that uses IBR techniques to approximate the integral lighting equation. Such an interactive algorithm is in demand now and will change the nature and work-flow of artists, designers and scientists.